Robust model selection with flexible trimming

We use the forward search to provide data-driven flexible trimming of the Cp statistic for the choice of regression models, thus revealing the effect of multiple outliers on model selection. We show, even in small sub-samples of the data, that is with heavy trimming, that the statistic virtually has a known null distribution a scaled and shifted F . We discuss the theoretical background to this form of distributional robustness. Our results on the null distribution provide a framework for informed robust model choice. Two examples of widely differing size are analysed. We develop a powerful graphical tool, the generalized candlestick plot, which summarises the information on forward searches and model choice for numerous models. A comparison of our method of flexible trimming is made with the use by Ronchetti and Staudte (1994) of M-estimation in robust model choice. The candlestick plot elucidates the properties of the two methods. In the related problem of the selection of non-nested models for Gaussian time series without regressors it is customary to use AIC. However, the distribution of this statistic is only known asymptotically. In order to provide informed model choice we accordingly derive a Cp-statistic, now with known asymptotic properties, which hold to an excellent approximation even in small samples. The statistic may be used for the choice of, for example, an ARMA time series model with Gaussian errors with, or without, regressor variables. The method can be applied not only to ARIMA models but also to the analysis of structural time series. As an example we analyse a time series of one hundred observations on the one-day ahead baseload price of electricity. A characteristic of such series is that they combine a stochastic process with explanatory variables. A complicated example is given by Koopman, Ooms, and Carnero (2007). We obtain a surprisingly rich understanding of potentially good and poor models. The known distribution of our statistic again greatly aids interpretation of our plots.